<!doctype html public "-//w3c//dtd html 4.0 transitional//en">
<html>
<head>
   <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
   <meta name="Author" content="Jeff W. Boote">
   <meta name="GENERATOR" content="Mozilla/4.79 [en] (Windows NT 5.0; U) [Netscape]">
   <title>BWCTL - Architecture</title>
<!--									-->
<!--	$Id$	-->
<!--									-->
<!------------------------------------------------------------------------>
<!--									-->
<!--			   Copyright (C)  2004				-->
<!--				Internet2				-->
<!--			   All Rights Reserved				-->
<!--									-->
<!------------------------------------------------------------------------>
<!--									-->
<!--	File:		architecture.html				-->
<!--									-->
<!--	Author:		Jeff Boote					-->
<!--			Internet2					-->
<!--									-->
<!--	Date:		Sat Feb 07 18:35:02 MST 2004			-->
<!--									-->
<!--	Description:							-->
<!--	Architecture description of bwctl.				-->
<!--									-->
</head>
<body>

<center>
<h1>
BWCTL Architecture</h1></center>

<center>
<h5>
$Date$</h5></center>
<h4>Components Overview</h4>
In the general case, BWCTL works by having a <b>bwctld</b> process on each
of the endpoint test systems. The <b>bwctl</b> client contacts each of
those daemons and requests a test. That request includes an indication
of the desired time period as well as a full parameterization of the desired
<b>Iperf</b>, <b>Nuttcp</b> or <b>Thrulay</b>
test. <b>bwctld</b> is responsible for implementing the policy and scheduling
constraints.
<center><h3>General Case</h3></center>
<center><h5><img SRC="bwctl_arch.png" height=501 width=601></h5></center>
<b>bwctld</b> has been developed as a classic accept/fork daemon. The master
daemon process listens for new network connections and also manages the
resources for all child <b>bwctld</b> processes. Once a connection comes
in, <b>bwctld</b> forks a child process to handle that request. The child
process deals with all encryption and communication issues with the client,
as well as dealing with all <i>static</i> resource limits. <i>Static</i>
resource limits are those not dependent upon what is currently happening
on the node. For example, the <i>request broker</i> can easily determine
if the given client is allowed to do UDP tests without talking to the <i>master
daemon</i>. Once the <i>request broker</i> process determines the request
is valid, it makes a request to the <i>master daemon</i> for the resources
and time period requested by the client. If the <i>master daemon</i> has
the resources available (including the open time period), it grants the
request.
<h4>Scheduling Requests</h4>
<p>The time period for the test is always requested by the client. It makes
a request to the <i>request broker</i> process on each endpoint of the
test in turn. The time period is requested by specifying two timestamps.
The earliest time and the latest time the client is willing to settle for.
The <i>request broker</i> either returns a tentative reservation with the
first open time that will fulfill the parameters of the test or a <b>request
denied</b> message. The client can then use the tentative reservation time
as the earliest time in the request to the <i>request broker</i> on the
other host. Eventually, an agreed upon time will be reached or the latest
time will be reached. If the <i>request broker</i> can not fulfill the
request before the latest time specified in the request, it will return
a <b>server too busy</b> message. If the client gets the same time from both servers,
it must then confirm the reservation with a <b>start session</b> message.
The <b>start session</b> message must be received before a configurable
timeout period and before the reservation time, or the server will disallow
the reservation.
<h4>Scheduling Algorithm</h4>
<p>The <b>bwctld</b> parent <i>resource broker</i> process holds the master
schedule. When requests come in from clients, the <i>request broker</i>
process does the non-global authorization checks and then passes the
global authorization checks up to the <i>resource broker</i> including
the request for a given time slot. The scheduling
itself is simply a first-fit algorithm with limits imposed on how far
it is willing to schedule into the future.</p>
<p>Each time slot that is requested is padded to allow for time synchronization
issues. The algorithm for the padding is dynamic and depends upon the
following factors:
<ul>
<dl>
    <dt>L (local)
    <dd>accuracy of the local clock as reported by NTP and the local
    sync_fuzz parameter setting
    <dt>P (peer)
    <dd>accuracy of the peer clock as reported by NTP and the peer
    sync_fuzz parameter setting
    <dt>R (round trip time)
    <dd>Round Trip Time between the local host and the peer
</dl>
</ul>
The algorithm is:<br>
<ul>A + L + P + (2 * R)</ul>
Where <b>A</b> is a small constant. (Currently, one second in the code.)
</p>
<p>
This amount of time is padded before and after each time slot that is
requested.
The algorithm was designed to continue to work in the case when
the test peer is not as well synchronized and is perhaps halfway around
the globe.</p>
<p>
The padding is placed before and after the test slot because there is
2 RTT of communication before and after each test.
For well synchronized hosts that are fairly near one another,
this algorithm usually provides 2-3 seconds of padding between each
test (time slot). This could potentially be reduced by making the
<b>A</b> constant smaller. That constant represents the amount of time it
takes a host to context switch, start processes, and includes how quickly
the throughput tester notices that its time is up. Experimentation has shown that even
with 1 second of padding, the throughput testers sometimes need to be killed off.</p>
<h4>Test Execution</h4>
<p>Once the client sends the the <b>start session</b> message, the <i>request
broker</i> forks off a <i>peer agent</i> that is responsible for verifying
the time offset to the other endpoint of the test and initializing the
communication socket that will be used to trade results of the test.
<p>If the test endpoint systems have a reasonably close idea of the time,
and they can communicate, the <i>peer agent</i> forks off the <i>test process</i>.
The <i>test process</i> waits until the scheduled start time and then executes
the desired throughput tester
with the correct command line parameters. Note: Experience is showing that
a closer integration with the testing process (<b>Iperf</b>, <b>Nuttcp</b> or <b>Thrulay</b>) may be needed
in the future.
<h4>End User Workstation Configuration</h4>
<p>In the event that the local host is one of the endpoints and there is
no local <b>bwctld</b> running, <b>bwctl</b> will spawn additional processes
to execute the local side of the test directly. In this case, a diagram
of the processes involved would look like the following:
<center><h3>Local bwctld not available</h3></center>
<center><h5><img SRC="bwctl_arch_servless.png" height=501 width=601></h5></center>
The same basic communication takes place between the different processes
in this alignment. The main difference is that the client has no need of
a resource broker. This is the expected usage scenario for end user workstations
to run tests to well defined BWCTL measurement beacons.
<p>* The images above shows a dashed blue line around the processes that
occur on each host.
<p>
<hr WIDTH="100%">
<br>$Id$
</body>
</html>
